49 research outputs found
Time and Frequency Domain Methods for Basis Selection in Random Linear Dynamical Systems
Polynomial chaos methods have been extensively used to analyze systems in
uncertainty quantification. Furthermore, several approaches exist to determine
a low-dimensional approximation (or sparse approximation) for some quantity of
interest in a model, where just a few orthogonal basis polynomials are
required. We consider linear dynamical systems consisting of ordinary
differential equations with random variables. The aim of this paper is to
explore methods for producing low-dimensional approximations of the quantity of
interest further. We investigate two numerical techniques to compute a
low-dimensional representation, which both fit the approximation to a set of
samples in the time domain. On the one hand, a frequency domain analysis of a
stochastic Galerkin system yields the selection of the basis polynomials. It
follows a linear least squares problem. On the other hand, a sparse
minimization yields the choice of the basis polynomials by information from the
time domain only. An orthogonal matching pursuit produces an approximate
solution of the minimization problem. We compare the two approaches using a
test example from a mechanical application
Enhancing adaptive sparse grid approximations and improving refinement strategies using adjoint-based a posteriori error estimates
In this paper we present an algorithm for adaptive sparse grid approximations
of quantities of interest computed from discretized partial differential
equations. We use adjoint-based a posteriori error estimates of the physical
discretization error and the interpolation error in the sparse grid to enhance
the sparse grid approximation and to drive adaptivity of the sparse grid.
Utilizing these error estimates provides significantly more accurate functional
values for random samples of the sparse grid approximation. We also demonstrate
that alternative refinement strategies based upon a posteriori error estimates
can lead to further increases in accuracy in the approximation over traditional
hierarchical surplus based strategies. Throughout this paper we also provide
and test a framework for balancing the physical discretization error with the
stochastic interpolation error of the enhanced sparse grid approximation
Generation and application of multivariate polynomial quadrature rules
The search for multivariate quadrature rules of minimal size with a specified
polynomial accuracy has been the topic of many years of research. Finding such
a rule allows accurate integration of moments, which play a central role in
many aspects of scientific computing with complex models. The contribution of
this paper is twofold. First, we provide novel mathematical analysis of the
polynomial quadrature problem that provides a lower bound for the minimal
possible number of nodes in a polynomial rule with specified accuracy. We give
concrete but simplistic multivariate examples where a minimal quadrature rule
can be designed that achieves this lower bound, along with situations that
showcase when it is not possible to achieve this lower bound. Our second main
contribution comes in the formulation of an algorithm that is able to
efficiently generate multivariate quadrature rules with positive weights on
non-tensorial domains. Our tests show success of this procedure in up to 20
dimensions. We test our method on applications to dimension reduction and
chemical kinetics problems, including comparisons against popular alternatives
such as sparse grids, Monte Carlo and quasi Monte Carlo sequences, and Stroud
rules. The quadrature rules computed in this paper outperform these
alternatives in almost all scenarios
Gradient-based Optimization for Regression in the Functional Tensor-Train Format
We consider the task of low-multilinear-rank functional regression, i.e.,
learning a low-rank parametric representation of functions from scattered
real-valued data. Our first contribution is the development and analysis of an
efficient gradient computation that enables gradient-based optimization
procedures, including stochastic gradient descent and quasi-Newton methods, for
learning the parameters of a functional tensor-train (FT). The functional
tensor-train uses the tensor-train (TT) representation of low-rank arrays as an
ansatz for a class of low-multilinear-rank functions. The FT is represented by
a set of matrix-valued functions that contain a set of univariate functions,
and the regression task is to learn the parameters of these univariate
functions. Our second contribution demonstrates that using nonlinearly
parameterized univariate functions, e.g., symmetric kernels with moving
centers, within each core can outperform the standard approach of using a
linear expansion of basis functions. Our final contributions are new rank
adaptation and group-sparsity regularization procedures to minimize
overfitting. We use several benchmark problems to demonstrate at least an order
of magnitude lower accuracy with gradient-based optimization methods than
standard alternating least squares procedures in the low-sample number regime.
We also demonstrate an order of magnitude reduction in accuracy on a test
problem resulting from using nonlinear parameterizations over linear
parameterizations. Finally we compare regression performance with 22 other
nonparametric and parametric regression methods on 10 real-world data sets. We
achieve top-five accuracy for seven of the data sets and best accuracy for two
of the data sets. These rankings are the best amongst parametric models and
competetive with the best non-parametric methods.Comment: 24 page
A generalized sampling and preconditioning scheme for sparse approximation of polynomial chaos expansions
In this paper we propose an algorithm for recovering sparse orthogonal
polynomials using stochastic collocation. Our approach is motivated by the
desire to use generalized polynomial chaos expansions (PCE) to quantify
uncertainty in models subject to uncertain input parameters. The standard
sampling approach for recovering sparse polynomials is to use Monte Carlo (MC)
sampling of the density of orthogonality. However MC methods result in poor
function recovery when the polynomial degree is high. Here we propose a general
algorithm that can be applied to any admissible weight function on a bounded
domain and a wide class of exponential weight functions defined on unbounded
domains. Our proposed algorithm samples with respect to the weighted
equilibrium measure of the parametric domain, and subsequently solves a
preconditioned -minimization problem, where the weights of the diagonal
preconditioning matrix are given by evaluations of the Christoffel function. We
present theoretical analysis to motivate the algorithm, and numerical results
that show our method is superior to standard Monte Carlo methods in many
situations of interest. Numerical examples are also provided that demonstrate
that our proposed Christoffel Sparse Approximation algorithm leads to
comparable or improved accuracy even when compared with Legendre and Hermite
specific algorithms.Comment: 32 pages, 10 figure
A Christoffel function weighted least squares algorithm for collocation approximations
We propose, theoretically investigate, and numerically validate an algorithm
for the Monte Carlo solution of least-squares polynomial approximation problems
in a collocation frame- work. Our method is motivated by generalized Polynomial
Chaos approximation in uncertainty quantification where a polynomial
approximation is formed from a combination of orthogonal polynomials. A
standard Monte Carlo approach would draw samples according to the density of
orthogonality. Our proposed algorithm samples with respect to the equilibrium
measure of the parametric domain, and subsequently solves a weighted
least-squares problem, with weights given by evaluations of the Christoffel
function. We present theoretical analysis to motivate the algorithm, and
numerical results that show our method is superior to standard Monte Carlo
methods in many situations of interest.Comment: 29 pages, 11 figure
Enhancing -minimization estimates of polynomial chaos expansions using basis selection
In this paper we present a basis selection method that can be used with
-minimization to adaptively determine the large coefficients of
polynomial chaos expansions (PCE). The adaptive construction produces
anisotropic basis sets that have more terms in important dimensions and limits
the number of unimportant terms that increase mutual coherence and thus degrade
the performance of -minimization. The important features and the
accuracy of basis selection are demonstrated with a number of numerical
examples. Specifically, we show that for a given computational budget, basis
selection produces a more accurate PCE than would be obtained if the basis is
fixed a priori. We also demonstrate that basis selection can be applied with
non-uniform random variables and can leverage gradient information
Compressed sensing with sparse corruptions: Fault-tolerant sparse collocation approximations
The recovery of approximately sparse or compressible coefficients in a
Polynomial Chaos Expansion is a common goal in modern parametric uncertainty
quantification (UQ). However, relatively little effort in UQ has been directed
toward theoretical and computational strategies for addressing the sparse
corruptions problem, where a small number of measurements are highly corrupted.
Such a situation has become pertinent today since modern computational
frameworks are sufficiently complex with many interdependent components that
may introduce hardware and software failures, some of which can be difficult to
detect and result in a highly polluted simulation result.
In this paper we present a novel compressive sampling-based theoretical
analysis for a regularized minimization algorithm that aims to recover
sparse expansion coefficients in the presence of measurement corruptions. Our
recovery results are uniform, and prescribe algorithmic regularization
parameters in terms of a user-defined a priori estimate on the ratio of
measurements that are believed to be corrupted. We also propose an iteratively
reweighted optimization algorithm that automatically refines the value of the
regularization parameter, and empirically produces superior results. Our
numerical results test our framework on several medium-to-high dimensional
examples of solutions to parameterized differential equations, and demonstrate
the effectiveness of our approach.Comment: 27 pages, 7 figure
Optimal Experimental Design Using A Consistent Bayesian Approach
We consider the utilization of a computational model to guide the optimal
acquisition of experimental data to inform the stochastic description of model
input parameters. Our formulation is based on the recently developed consistent
Bayesian approach for solving stochastic inverse problems which seeks a
posterior probability density that is consistent with the model and the data in
the sense that the push-forward of the posterior (through the computational
model) matches the observed density on the observations almost everywhere.
Given a set a potential observations, our optimal experimental design (OED)
seeks the observation, or set of observations, that maximizes the expected
information gain from the prior probability density on the model parameters. We
discuss the characterization of the space of observed densities and a
computationally efficient approach for rescaling observed densities to satisfy
the fundamental assumptions of the consistent Bayesian approach. Numerical
results are presented to compare our approach with existing OED methodologies
using the classical/statistical Bayesian approach and to demonstrate our OED on
a set of representative PDE-based models
A Generalized Approximate Control Variate Framework for Multifidelity Uncertainty Quantification
We describe and analyze a variance reduction approach for Monte Carlo (MC)
sampling that accelerates the estimation of statistics of computationally
expensive simulation models using an ensemble of models with lower cost. These
lower cost models --- which are typically lower fidelity with unknown
statistics --- are used to reduce the variance in statistical estimators
relative to a MC estimator with equivalent cost. We derive the conditions under
which our proposed approximate control variate framework recovers existing
multi-model variance reduction schemes as special cases. We demonstrate that
these existing strategies use recursive sampling strategies, and as a result,
their maximum possible variance reduction is limited to that of a control
variate algorithm that uses only a single low-fidelity model with known mean.
This theoretical result holds regardless of the number of low-fidelity models
and/or samples used to build the estimator. We then derive new sampling
strategies within our framework that circumvent this limitation to make
efficient use of all available information sources. In particular, we
demonstrate that a significant gap can exist, of orders of magnitude in some
cases, between the variance reduction achievable by using a single low-fidelity
model and our non-recursive approach. We also present initial sample allocation
approaches for exploiting this gap. They yield the greatest benefit when
augmenting the high-fidelity model evaluations is impractical because, for
instance, they arise from a legacy database. Several analytic examples and an
example with a hyperbolic PDE describing elastic wave propagation in
heterogeneous media are used to illustrate the main features of the
methodology